A deep-hole observation method using a charged particle beam apparatus such as a scanning electron microscope (SEM) is known. PTL 1 discloses an SEM having two stages of detectors by which electrons (electrons (high angle electrons) passing through a trajectory having a relatively small angle with respect to the optical axis) passing through a trajectory near the optical axis of the electron beam and electrons (electrons (low angle electrons) passing through a trajectory having a relatively large angle with respect to the optical axis relative to the high angle electrons) passing through a trajectory relatively separated from the optical axis of the electron beam are discriminated and detected in order to form a high-contrast hole bottom image based on the electrons emitted from the bottom of a deep hole such as a contact hole. PTL 2 and PTL 3 disclose the SEM in which a deflector for secondary electrons used for controlling the trajectory of secondary electrons is provided so as to make detection signals of the plurality of detectors uniform.